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CROSS-REFERENCE TO RELATED APPLICATIONS 

This application claims priority under 35 U.S.C. §1 19(e) from co-pending (1) 
U.S. Provisional Application No. 60/395,357, entitled "Method and Apparatus for 3-D 
Imaging of Internal Light Sources", by Daniel G. Stearns, et al., (2) U.S. Provisional 
Application No. 60/396,458, entitled "In V ivo 3D Imaging of Light Emitting 
Reporters", by Bradley W. Rice, et al. and (3) U.S. Provisional Application No. 
60/396,313, entitled "3D in Vivo Imaging of Light Emitting Reporters", by Bradley W. 
Rice, et al. These applications were all filed on July 16, 2002 and are incorporated by 
reference for all purposes. 

FIELD OF THE INVENTION: 

The present invention relates generally to imaging with light. In particular, the 
present invention relates to systems and methods for obtaining a three-dimensional 
representation of a light source or light source distribution inside a turbid sample, which 
is particularly useful in biomedical imaging and research applications. 

BACKGROUND OF THE INVENTION: 

Bioluminescent imaging is a non-invasive technique for performing in vivo 
diagnostic studies on animal subjects in the areas of medical research, pathology and 
drug discovery and development. Bioluminescence is typically produced by cells that 
have been transfected with a luminescent reporter such as luciferase and can be used as a 
marker to differentiate a specific tissue type (e.g. a tumor), monitor physiological 
function, track the distribution of a therapeutic compound administered to the subject, or 
the progression of a disease. A wide range of applications have been demonstrated 
including areas of oncology, infectious disease, and transgenic animals. In vivo imaging 
of cells tagged with fluorescent reporters is a related technology that has also been 
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general, absorption in mammalian tissues is high in the blue-green part of the spectrum 
(<600 nm) and low in the red and NIR part of the spectrum (600 - 900 nm). Firefly 
luciferase has a rather broad emission spectrum ranging from 500 - 700 nm, so at least 
part of the emission is in the low absorption region. Since the mean-free-path for 
scattering in tissue is short, on the order of -0.5 mm, photons from deep sources are 
scattered many times before reaching the surface. Bioluminescent imaging systems 
effectively record the spatial distribution of these photons emitted from the surface of the 
subject. 

However, the most important quantitative information is not directly related to 
the surface emission but instead pertains to the bioluminescent source inside the subject. 
Important parameters are the source strength (related to the number of light emitting 
cells), position and geometry. Most of the bioluminescent imaging work published to 
date involves use of single-view 2D imaging systems. Image analysis usually involves 
quantifying a light emitting region-of-interest (ROI) on the subject surface. While this 
analysis methodology is simple and provides a good relative measure of light emission, 
it does not take into account the source depth and resulting attenuation through tissue. 

Hence, there is interest in developing both improved imaging systems and 
reconstruction algorithms that would provide the three-dimensional distribution of 
photon emission inside the sample (e.g., animal) from images measured on the sample 
surface. 

SUMMARY OF THE INVENTION 

The present invention provides systems and methods for obtaining a three- 
dimensional (3D) representation of one or more light sources inside a sample, such as a 
mammal. Mammalian tissue is a turbid medium, meaning that photons are both 
absorbed and scattered as they propagate through tissue. In the case where scattering is 
large compared with absorption, such as red to near-infrared light passing through tissue, 
the transport of light within the sample is described by diffusion theory. Using imaging 
data and computer-implemented photon diffusion models, embodiments of the present 
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invention produce a 3D representation of the light sources inside a sample, such as a 3D 
location, size, and brightness of such light sources. 

In one aspect, the present invention relates to a method for obtaining a three- 
dimensional representation of a light source distribution located inside a turbid sample. 

5 Surface light image data is provided from light emitted from a surface of the sample 
originating from the light source distribution located inside the sample. Light data 
internal to the sample surface is then obtained based on this provided surface light image 
data. In a specific embodiment, the surface light image data is converted into photon 
density just below the sample surface, and this photon density is then used to obtain the 

10 light data internal to the sample surface to thereby result in a three-dimensional 
representation of the light source distribution within the sample. 

In yet another aspect, the present invention relates to an imaging system for 
obtaining a three-dimensional representation of a light source located inside a sample. 
The imaging system comprises an imaging chamber having a set of walls enclosing an 

15 interior cavity. The imaging chamber also includes a camera mount configured to 
position a camera and a moveable stage apparatus. The movable stage apparatus 
includes a transport mechanism and a stage configured to support the sample within the 
interior cavity. The stage is coupled with the transport mechanism for movement of the 
sample to one of a plurality of positions in the interior cavity. The imaging chamber 

20 further includes a light transport mechanism for transmitting light emitted from a surface 
of the sample. The imaging chamber also includes a processor designed or configured to 
provide surface light image data from light emitted from the surface of the sample 
originating from the light source distribution located inside the sample. The processor is 
further configured to obtain light data internal to the sample surface to obtain a three- 

25 dimensional representation of the light source distribution based on the surface light 
image data. 

In still another aspect, the present invention relates to a computer program 
product. The computer program product comprises a computer readable medium and 
program instructions provided via the computer readable medium. The program 
30 instructions comprise reconstruction instructions for obtaining a three-dimensional 

representation of a light source located inside a sample. The reconstruction instructions 
are capable of performing one or more of the above described method operations. 



XENOP009/PXT- -01 1 TTS 



These and other features of the present invention will be described in more detail 
below in the detailed description of the invention and in conjunction with the following 
figures. 

BRIEF DESCRIPTION OF THE DRAWINGS 

The present invention is illustrated by way of example, and not by way of 
limitation, in the figures of the accompanying drawings and in which like reference 
numerals refer to similar elements and in which: 

FIG. 1 is a perspective view of an imaging system adapted to produce a 3D 
representation of a light source located inside a sample in accordance with one 
embodiment of the invention. 

FIG. 2 A is a cut away perspective view of the chamber of FIG. 1 having internal 
components for facilitating multiple views of the sample in accordance with one 
embodiment of the present invention. 

FIG. 2B is a perspective view of the internal components of FIG. 2A for 
facilitating multiple views of the sample in accordance with one embodiment of the 
present invention. 

FIG. 2C shows four different example positions of the stage relative to the light 
transport device of FIGs. 2A and 2B: 0 degrees, 90 degrees, 180 degrees, and 270 
degrees. 

FIG. 2D shows the configuration of the structured light projector used to obtain 
the subject surface topography 

FIG. 3 A shows a simplified illustration of light imaging in a turbid medium in 
accordance with one embodiment of the present invention. 

FIG. 3B illustrates a schematic diagram showing a planar approximation at the 
surface boundary. 

FIG. 3C illustrates the relationship between a volume element (voxel) and a 
surface element for a cylindrical object. 

4 * 
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FIG. 4A illustrates exemplary relationships for converting between 2D camera 
data and surface data for an exemplary surface element. 

FIGs. 4B-4H illustrate pictorial representations of structured light imaging 
corresponding to the process flow of FIG. 5C. 

5 FIG. 5 A illustrates a process flow for obtaining a 3D representation of one or 

more light sources inside a sample in accordance with one embodiment of the present 
invention. 

FIG. 5B illustrates a process flow for using the imaging system of FIG. 1A to 
obtain imaging data in accordance with one embodiment of the present invention. 

FIG. 5C illustrates a process flow for using the imaging system of FIG. 1 A to 
obtain surface topography data in accordance with one embodiment of the present 
invention. 

FIG. 5D illustrates a process flow for converting surface light emission data to 
light internal to the surface in accordance with a specific embodiment of the present 
invention. 

FIGs. 5E and 5F represent reconstruction results for one point-like light source 
within a phantom mouse {e.g., plastic mouse having an embedded fiber optic). 

r 

FIG. 6 illustrates an exemplary computer system in accordance with one 
embodiment of the present invention. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 

In the following detailed description of the present invention, numerous specific 
embodiments are set forth in order to provide a thorough understanding of the invention. 
However, as will be apparent to those skilled in the art, the present invention may be 
25 practiced without these specific details or by using alternate elements or processes. In 
other instances well known processes, components, and designs have not been described 
in detail so as not to unnecessarily obscure aspects of the present invention. 
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1. Imaging System 

The 3D light reconstruction techniques of the present invention may be 
implemented within any suitable system. Several embodiments of imaging systems in 
which are suitable for implementing the techniques of the present invention are 
5 described further in U.S. Patent Application No. 09/905,668 filed by Nilson et al. on July 
13, 2001, entitled MULTI-VIEW IMAGING APPARATUS. The entire disclosure of 
this application is incorporated herein by reference for all purposes. 

FIG. 1 illustrates an imaging system 10 adapted to produce a 3D representation 
of a light source located inside a sample in accordance with one embodiment of the 
10 present invention. The imaging system 10 may be used for a variety of tasks. By way of 
examples, these tasks may include capturing photographic, luminescent, and structured 
light images. These tasks may also include performing various processing and imaging 
tasks such as obtaining, analyzing and manipulating 3D light source representations. 

The system 10 generally images and processes low intensity light sources using a 
15 light sensitive camera. Light from the light source refers to photons and electromagnetic 
energy anywhere in the visible to near-infrared (NIR) part of the spectrum in the 
wavelength range of 400-950 nm. It is understood that some intensities imaged and 
processed in system 10 are not readily detectable by human vision. For example, low 

4 

intensity light emitted from a sample may have a surface radiance between about 10 3 to 
20 about 10 10 photons/second/centimeter squared/steradian, where the lower end of this 
range is typically well below human detection levels. 

In the illustrated embodiment, the imaging system 10 includes an imaging 
chamber 12 adapted to receive a light-emitting sample in which low intensity light, e.g., 
luciferase-based luminescence, is to be detected. A high sensitivity camera 20, such as 
25 an intensified or a charge-coupled device (CCD) camera, is coupled with the imaging 
chamber 12. The camera 20 is capable of capturing luminescent, photographic (i.e., 
reflection based images) and structured light images of the sample within imaging 
chamber 12. The camera 20 may optionally be cooled by a suitable source such as a 
refrigeration device 22 that cycles a cryogenic fluid through the camera via conduits 24. 
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An image processing unit 26 optionally interfaces between camera 20 and a 
computer 28 through cables 30 and 32, respectively. The computer 28, which may be of 
any suitable type, typically comprises a main unit 36 that contains hardware including a 
processor, memory components such as random-access memory (RAM) and read-only 
5 memory (ROM), and disk drive components (e.g., hard drive, CD, floppy drive, etc.). 
The computer 28 also includes a display 38 and input devices such as a keyboard 40 and 
mouse 42. The computer 28 is in communication with various components in the 
imaging chamber 12 via cable 34. Alternatively, a computer may be integrated with the 
imaging equipment in a single unit. 

10 To provide control and communication for components in system 10, computer 

28 may be implemented with any suitable combination of hardware and software for 
controlling and monitoring any of the devices in system 10. Components controlled by 
the computer 28 may include camera 20, the motors responsible for camera 20 focus, the 
camera lens, f-stop, one or more motors coupled to a moveable stage included in 

15 chamber 12, etc. Computer 28 also communicates with a display 38 for presenting 

imaging information to the user and also acts an interface to control the imaging system 
10. 

Computer 28 also includes suitable processing hardware and software for image 
processing and data manipulation as described herein. For example, computer 28 may 

20 be configured with software to build a 3D representation of a light source using light 

emission measurements from the surface of a sample. In addition, the computer 28 may 
be configured to produce 3D surface topography using 2D structured light images taken 
from one or more positions of the stage in the interior cavity. These 3D light 
construction techniques are further described below. Alternatively, the computer 28 may 

25 be integrated with the other components of the imaging system 10 to form a stand alone 
device. 

In one implementation, the 3D light reconstruction techniques discussed further 
below require bioluminescent surface radiance and surface topography measurements 
over the entire sample surface. Thus, the imaging system also includes mechanisms for 
30 capturing images of the sample surface from a variety of views and positions. Preferably 
the imaging system meets the following goals. First, it is desirable for the camera to 
remain fixed, because the various cables and supply lines for cooling the camera are too 
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cumbersome to move during imaging. Second, the sample preferably remains horizontal 
during imaging. Finally, the imaging system is preferably configured to obtain a 3D 
surface topography of the sample as input to the reconstruction techniques described 
further below. Any suitable combination of hardware and software may be used to 
5 implement these preferred goals. 

FIGs. 2A and 2B illustrate various internal components of chamber 12 for 
facilitating multiple view imaging in accordance with one embodiment of the present 
invention. FIG. 2A is a cut away perspective view of chamber 12 and its internal 
components, while FIG. 2B is a perspective view of the internal components of FIG. 2 A 
10 in accordance with one embodiment of the present invention. In the illustrated 

embodiments, the sample is placed on a horizontal transparent platform that rotates in 
conjunction with a mirror that turns about a horizontal optical axis so as to capture light 
from the sample from various views and direct the captured light towards a camera for 
imaging. 

15 As shown, a moveable stage apparatus is disposed in the interior cavity of 

chamber 12 (FIG. 2A) in the form of a stage 204 to support the light-emitting sample 
106 and a transport mechanism 202 to move the stage, A light transport device 120 is 
included to collect light from the sample at its various positions. In the illustrated 
embodiments of FIGs. 2 A and 2B, light emitted from the sample in the form of light 

20 beam 252a generally reflects off light transport device 120 as beam 252b. The light 
transport device directs this reflected light 252b through aperture 122 and through lens 
262 (FIG. 2B) for image capture by a detector 264 (FIG. 2B) of camera 20 (FIG. 1). 

In one implementation, light transport device 120 includes an angled mirror 121 
that reflects light from the sample 106 on stage 204 through aperture 122. Outer wall 

25 123 is substantially cylindrical and includes aperture 122 that enables light to pass from 
the sample 106 on stage 204 via mirror 121 to imaging lens 262 (FIG. 2B). Outer wall 
123 of light transport device 120 also prevents residual light in interior cavity of 
chamber 12 not directly associated with the current viewing angle of stage 204 from 
reaching lens 262. This is partially performed by configuring mirror 121 to be 

30 sufficiently long to span the length of stage 204. As the stage is positioned at various 
positions about the stationary axis of the light transport device 120; outer wall 123 and 
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mirror 121 cooperate to collect light primarily from the angular direction of stage 204 
which is then reflected towards lens 262. 

In the illustrated embodiment, light transport device 120 is rotably coupled to 
chamber 12 and rotates about a fixed axis in direction 258. The moveable stage 

5 apparatus is capable of two degrees of freedom to position the stage 204 (and sample 
106) at a plurality of positions relative to the light transport device 120 so as to capture 
light from different sides of the sample to thereby form multiple view images of the 
sample. The moveable stage apparatus allows an image of the sample, or portions 
thereof, to be captured by a camera from different views, angles, and positions within the 

10 imaging chamber without repositioning the posture of the sample relative to the stage 
204. FIG. 2C shows four different example positions of the stage relative to the light 
transport device (120): 0 degrees, 90 degrees, 180 degrees, and 270 degrees. 

The movable stage apparatus may be formed from any suitable number, type, and 
arrangement of components for achieving multiple positioning and viewing of the 
15 sample. Several embodiments of a multiple- view imaging systems are described in the 
above referenced patent application by Nilson et al, which embodiments are incorporated 
herein by reference. Preferably, the movable stage apparatus also includes sensor 
mechanisms for sensing a position of the sample to be used in subsequent data analysis 
procedures as described further below. 

20 In one embodiment, a light source is provided within the barrel of mirror 

assembly 120 (not shown) to illuminate the sample or specimen in the imaging chamber 
12. The light source may be continuously illuminated or flashed to capture photographic 
images of the sample and is turned off when capturing luminescence images. 

A structured light illuminator is preferably integrated into the system for 
25 reconstructing a surface topography of the sample. In a general implementation, the 
imaging system preferably includes a mechanism for projecting a grid of lines onto the 
sample from an angle, e.g., from 20 to 30 degrees from normal. The grid of lines are 
displaced, or phase shifted relative to the platform, when they encounter an object with 
finite height. Using conventional equations for structured light analysis, the surface 
30 topography data for the sample over its entire surface can be determined from the phase 
shift of the lines. The line spacing may vary based on the sample surface texture and 



sample size, but line spacing in the range of 0.5 to 2 lines per mm may be suitable for a 
mouse sample. Closer line spacing provides higher resolution, but the lines are more 
difficult to track on rough surfaces such as fur. The surface topography data is used to 
build a 3D representation according to the techniques described below, e.g., to set 
5 boundary conditions for the 3D reconstruction. 

In a specific embodiment, a simple Kohler projection scheme is used as the 
structured light source. In this case, the ruling may be illuminated by a diffuse LED 
source and the ruling is then projected onto the animal stage with a magnification of 
approximately lOx. An example of this system as incorporated in system 10 is shown in 
10 FIG. 2D. The projector module 170 rides on the back of the rotating mirror assembly 
120, so that lines are always projected on the sample 106 at all viewing angles. The 
illumination pattern is projected horizontally and reflects off of a small mirror 173 at the 
base of the larger turning mirror to illuminate sample 106. 

Such an imaging system as described with respect to FIGs. 2A-2D is available 
15 from Xenogen Corporation of Alameda, CA. Although FIGs. 2A-2D illustrate one 
system for obtaining data from multiple positions of sample 106 relative to camera 20, 
the present invention is not limited to such a system. 

2. 3D Source Reconstruction 

* 

In many turbid samples such as a mammalian subject, light traveling from a light 
20 source inside the sample and emitted from the sample surface is scattered in the sample 
interior (e.g., the mammal tissue) such that light propagation through the sample is 
diffusive in nature (Fig. 3A). As the light diffuses through the sample, some of the light 
is absorbed, but a fraction of the light reaches the sample surface. 

Emission of light from a sample surface is generally specified in units of 
25 radiance, defined as photons/sec/cm 2 /steradian. The imaging system described herein is 
calibrated to report surface intensity in units of radiance. The surface radiance can be 
related to the photon density just inside the sample surface, using a model for photon 
propagation at the tissue-air interface. The photon density just inside the surface can 
then be related to the distribution of light emitting reporters inside the sample using a 
30 diffusion model. Thus, the present invention relates the surface radiance of a turbid 
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sample measured with an imaging system to the distribution of light emission inside the 
sample. More specifically, the present invention produces a 3D representation of an 
internal light source using reconstruction techniques that utilize the light data emitted 
from the sample surface. The reconstruction techniques employ an input data set that 

5 consists of a) a topographical surface representation of the sample, and b) a set of 
measurements (e.g. surface images) of the light radiance-over at least a portion of the 
surface. To facilitate processing, the surface representation may be divided into surface 
elements and the interior of the sample may be divided into volume elements or voxels 
that constitute a volume element mesh. The light source distribution within the sample 

10 is described by elements of the volume element mesh. 

FIG. 3 A shows a simplified illustration of light imaging in accordance with one 
embodiment of the present invention. FIG. 3A displays a mammalian sample 302 being 
imaged by an imaging system 10 that uses CCD camera 20 for light data capture. The 
nature of mammalian tissue and many other turbid media causes photon scattering 307 
15 for light traveling from a light source 305 within the sample to the sample surface 309, 
resulting in diffuse images on surface 309. Using photon diffusion models and imaging 
data, the present invention produces the 3D location, size, and brightness of light source 
305 from one or more surface 309 images. For user convenience, the 3D representation 
may be expressed as a pictorial depiction, e.g., via display 38 in FIG. 1. 

20 One useful application of the invention is to reconstruct one or more 

bioluminescent or fluorescent light sources, or a source distribution, inside a living 
animal subject, such as a mouse, using one or more images of the surface light emission. 
This finds use in oncology, infectious disease research, gene expression research, and 
toxicology, for example. The present invention is suitable for use with samples having a 

25 complex surface, such as an animal subject (e.g. a mouse). As the term is used herein, a 
complex surface is any surface that cannot be described solely using a single polygonal 
description. The reconstruction techniques described herein place no restrictions on the 
source distribution, such as the number of light sources in the sample or the sizes and 
shapes of the sources, and no restrictions on the geometry, size or shape of the surface. 

30 Light data internal to the sample surface generally refers to mathematical 

representation or approximation of actual light within the sample interior. This may 

4 

include a set of points or volume elements, each characterized by 3D position and a 
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source strength. In one embodiment, the present invention divides the sample interior 
into volume elements where each volume element is considered to contain a point light 
source at its center. A solid mesh of these volume elements then defines a collection of 
point sources used to approximate the actual light source distribution within the sample. 
5 For example, a solid mesh of cubic volume elements is suitable. 

FIG. 3C illustrates an illustrative cylindrical object that is divided into a solid 
mesh of volume elements, one of which is shown. Each volume element contains a 
point light source 314, Si, that contributes to the photon density 316, Pj, at the surface. 
For a cubic volume element representation, the volume element size may vary according 
10 to the size of the sample and a desired solution accuracy. In general, increasing the solid 

mesh density improves the accuracy of the internal light data representation. It is 

» 

possible to have the density of the solid mesh vary with the volume of the sample. In 
particular, it is desirable to have the solid mesh density increase near the light source to 
provide a more accurate representation in this region, while density of the solid mesh 
15 decreases in areas where no activity of interest is taking place (in areas corresponding to 
no or minimal light generation). 

The surface representation refers to the surface topography of the sample and is 
embodied by a mathematical description or approximation of the actual surface. The 
surface representation need not include the entire sample, and may include a portion of 
20 the sample relevant to a particular imaging scenario. With a mouse for example, the 
surface representation might not necessarily include the entire mouse such as distal 
portions of the tail and distal portions of every foot. Thus, the surface representation is 
meant to broadly refer to any surface portion of the sample and not necessarily the entire 
sample. Typically, the surface representation includes one or more surface elements or 

9 

25 regions of interest on the sample that produce surface light emission data related to the 
internal light source. For user convenience, the surface representation is often displayed 
in a pictorial depiction such as a 3D depiction derived from structural light construction 
techniques. 

The sample containing the light source comprises a turbid interior of some sort. 
30 A turbid interior in this case refers to a volume that does not allow unimpeded transport 
of light. The turbid interior may comprise one or more mediums, structures, solids, 
liquids, gases, etc. In one embodiment, the sample is modeled as homogeneous such 
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that each representative volume element in the sample is characterized by identical light 
transport properties. In another embodiment, the sample is represented as heterogeneous 
such that various representative volume elements in the sample are characterized by 
different light transport properties. In a mammalian sample such as a mouse for 
5 example, the interior may comprise a mixture of tissues, bones, organs, etc., each of 
which may be characterized by separate light transport properties in a heterogeneous 
model. 

The present invention presents several methods for reconstructing a three- 
dimensional light source distribution internal to a sample based on the surface light 

10 emission, as will be described in greater detail below. Briefly, a set of structured light 
and luminescent images are first acquired, e.g., obtained with a system as described 
above. The surface topography of the animal is reconstructed using structured light 
algorithms and a surface mesh is generated. Let the surface elements be enumerated with 
the integers j. Using one or more luminescent images, the orientation of the imaging 

15 system, and the orientation of the surface mesh, the photon density, Pj, just below the 
surface of the element can be determined. The photon density just below the surface 
is related to the light intensity emitted from the surface (Eq. 1) as will be detailed below. 
A set of volume elements can be constructed in the volume interior to the surface. Let 
the volume elements be enumerated with the integers i. The source strength in each 

20 volume element, Si, is linearly related to the photon density just below the surface 

through a Green's function kernel, Gij , yielding a set of linear equations (Eq. 4) as will 
be detailed below. 

FIG. 5 A illustrates a process flow 500 for obtaining a 3D representation of one or 
more light sources inside a sample in accordance with one embodiment of the present 
25 invention. Processes in accordance with the present invention may include up to several 
additional steps not described or illustrated herein in order not to obscure the present 
invention. 

Process flow 500 uses 3D reconstruction techniques that manipulate light data 
emitted from a sample surface. The reconstruction techniques employ data that provides 
30 information about the geometry of the sample and the spatial distribution of the light 
emission from the sample surface. This may include data that comprises a) a surface 

i 
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representation of the sample, and b) a set of measurements (e.g. images) of the light 
emission over at least a portion of the surface. One format for this data comprises (1) a 
surface representation defining the surface of the sample, (2) a set of light emission 
images from the sample surface, and (3) a set of parameters that define the spatial 
relationship between the sample and each image. 

Process flow 500 begins by obtaining input data (502 and 504) used in the 3D 
reconstruction techniques. For example, process flow 500 may begin by obtaining a 
surface representation (502). To facilitate processing, the surface representation may be 
divided into surface elements or a suitable surface mesh approximating the actual 
surface of the sample (502). For example, the surface mesh may be defined by a set of 
connected polygons, where each polygon is a surface element. The number of surface 
elements may vary according to the size of the sample and a desired solution accuracy. 
Also the density of surface elements may vary from point to point on the surface mesh. 
In general, the accuracy of a surface representation is improved by increasing the number 
of surface elements. 

Process flow may also obtain surface light emission data from the sample (504). 
For imaging system 10, the surface light emission data is contained in images captured 
by camera 20. The images include a set of measurements of the light emission over one 
or more portions of the surface. In one embodiment, multiple images of the sample are 
obtained from multiple views of the sample relative to the camera. In this case, each 
image provides a different two-dimensional view of the surface light emission from the 
sample. Multiple images may be taken to provide additional data from multiple angles. 

The image data may then be mapped back onto the surface of the sample. Since 
a camera produces 2D data, the image data is manipulated according to the geometry 
between the sample surface and the camera lens to derive values of the light emission 
intensity (or radiance) at the surface. 

FIG. 4A is an illustrative relationship for converting 2D camera data to surface 
data for an exemplary surface element 402. FIG. 4A shows the relationship between 
surface element 402, image 404, and an entrance pupil 406 of the imaging system. The 
light emitted from surface element 402 passes through entrance pupil 406 and is 
recorded in image 404. The angle of emission with respect to the surface normal is 02. 



1 



The entrance pupil 406 subtends a small solid angle d£2. Imaging system 10 may collect 
light emitted from a surface element 402 on the sample at an angle 02 (measured with 
respect to the normal to surface element 402) into the solid angle d£2 subtended by the 
entrance pupil. This information may then be used to convert image data obtained by the 
5 camera into the surface emission intensity corresponding to the surface geometry. 

Process flow 500 then converts the surface light emission data into light data 
internal to the surface to obtain the three-dimensional representation (506). The surface 
emission intensity is related to the photon density just inside the sample surface. The 
present invention thus converts values of light emission intensity for each surface 

10 element into photon density just inside the surface. Referring again to FIG. 4A, the 
value of emission intensity at a surface element, 1(62), is related to the photon density p 
beneath the surface element. The exact form of the relationship depends on the model 
used to describe the transport of photons across the surface boundary. One embodiment 
of this relationship, based on the partial-current boundary condition [Haskell, et al.], is 

15 given by: 



c 



/(6L) = - T T{0)cos0 2 dQ l + ~ ^-cos0 



Ann 



(1) 



Here, c is the speed of light, n is the index of refraction of the sample medium, T is the 
transmission coefficient for light exiting the sample through the surface element, and 0 is 
the internal emission angle, which is related to the external emission angle ©2 through 
20 Sneirs law: 

n sin 0 = sin 0 2 (2) 

The parameter R e ff is the average internal reflection coefficient calculated from 
the following formulae: 
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R^ = |2 2sin0cos9R(9)d9 
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Rj = j2 3sin9cos 2 0R(0)d9 
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ncos9-cos9 2 
^ncos9 + cos8 2 , 
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for9<arcsin(l/n) 



Thus, the internal reflectivity R e ff depends on the index of refraction of the 
medium underneath a surface element. In tissue for example, R e ff is typically in the 
5 range of 0.3 - 0.5. 

Eqs. (1) and (2) may thus be used to convert surface emission data measured at 
each surface element to values of the photon density beneath the surface. As the term is 
used herein, the subscript j enumerates a set of surface elements. Pj is then the value of 
the photon density calculated at the jth surface element. Further description of the 
10 conversion between surface light emission data into light data internal to the surface to 
obtain the three-dimensional representation is provided in FIG. 5D. 

FIG. 5B illustrates a process flow 520 for using imaging system 10 of FIG. 1 to 
obtain imaging data in accordance with one embodiment of the present invention (504 
from process flow 500). Process flow 520 begins by placing a sample such as a 

15 specimen or assay to be imaged for light emission on stage 204 within imaging chamber 
12 (521). Using computer 28, a user inputs a desired position for stage 204. 
Alternatively, the desired position for stage 204 is already known based on an automated 
data collection method. Transport mechanism 202 moves stage 204 to the desired 
position according to a control signal provided by computer 28 (522). Light transport 

20 device 120 may also re-position according to a control signal provided by computer 28. 
Imaging chamber 12 and associated image components are then prepared for 
photographic image capture of the sample (523). Preparation may include launching 
imaging and acquisition software (e.g., "Livinglmage" as provided by Xenogen 
Corporation of Alameda, CA) on computer 28 and initializing camera 20. Further 
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preparations may include closing door 18, activating the photographic capture option in 
the software, focusing camera 20 to a specific depth of the sample or animal, and turning 
on the lights in chamber 12. Preparations may also include focusing the lens of camera 
20, selectively positioning an appropriate the lens filter of camera 20, setting the f-stop 
of camera 20, etc. 

A photographic image is then captured (524). In an alternative embodiment, a 
"live mode" is used during photographic imaging of the sample to observe the sample in 
real time. The live mode includes a sequence of photographic images taken frequently 
enough to simulate live video. Upon completion of photographic capture, the 
photographic image data is transferred to an image processing unit 26 and/or a processor 
in computer system 28. These may be used to manipulate and store the photographic 
image data as well as process the data for display on computer monitor 38. 

In one embodiment, the present invention uses structured light during image 
capture. Thus, with stage 204 at a desired position, a structured light image may be 

i 

captured (526). Structured light image capture may be accomplished using a structured 
light projection system as described above, and may also include preparation of the 
structured light projection system (525) and any other components of imaging system 10 
used in the structured light image capture. Upon completion, the structured light image 
data is transferred to an image processing unit 26 and/or a processor in computer system 
28, which may be useful in building a 3D representation of the sample. 

Subsequently, with stage 204 at a desired position or at the same position as a 
previously taken photographic or structured light image, the imaging apparatus 10 is 
prepared for luminescence image capture (527). Such preparation may include selecting 
luminescent exposure time and binning level using the computer 28, and turning off the 
lights inside chamber 12. When ready, the CCD camera 20 then captures (514) the 
luminescence image over a set period of time (up to several minutes). The luminescence 
image data are transferred to the image processing unit and/or a processor in computer 
28. 

* 

At this point, a user may manipulate and store the luminescence image data as 
well as process it for display on the computer display 38. The manipulation may also 
include overlaying the luminescent image with the photographic image and displaying 
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the two images together as a 2-D "overlay" image , with the luminescence data typically 
shown in pseudocolor to show intensity. As mentioned, the photon emission data may 
represent the specific pixels on the camera 20 that detect photons over the duration of the 
image capture period. This overlay image may then be the basis for user analysis; and 

5 may be analyzed and manipulated as desired. In particular, an analysis may include a 
summation of the illumination magnitudes over the pixels within a portion of the 
luminescence representation. Note that although the discussion will focus on a single 
luminescence representation for the overlay image, the process flow 520 may include 
taking multiple luminescence representations from the same position of stage 204, e.g., 

10 * at the same time or a later time (530). 

Stage 204 may then be moved to a second position (531). While the stage is at 
the second position, one or more photographic, structured light, and/or luminescence 
images of the sample may be captured as described above. Image collection may further 
continue by capturing images of the sample from alternate positions and views of the 
15 sample. For 3D reconstruction, photographic, structured light, and/or luminescence 

images of the sample may be captured from a number of positions. For example, image 
capture may occur at anywhere from 2 to 200 positions of the sample within the imaging 
chamber. The preferred number of images is 8, spaced every 45 degrees. In one 
embodiment, this process is automated and controlled via computer 28. 

20 After a suitable number of structured light images have been captured from 

various angles and stored, computer 28 may then process the structured light data from 
each angle to generate a structured light representation (532). Each structured light 
image provides the surface topography for approximately the facing half of the sample. 

In another embodiment, process flow 520 and imaging apparatus 10 reconstruct 
25 the 3D surface topography of the sample using a sequence of images. By taking images 
from several viewing angles, e.g., about every 45 degrees, the entire 3D surface of the 
sample can be reconstructed by "stitching" together the partial surface reconstructions 
obtained from each view. A sequence of images may then be taken at different viewing 
angles and used in reconstructing the sample's 3D surface topography. The 3D surface 
30 topography and image data may also be used in reconstructing the 3D location, 

brightness, and size of the light source within the sample. Once the images are received 
by processor 28, a suitable reconstruction algorithm is applied to the data to obtain the 
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3D surface topography. As one of skill in the art will appreciate, there are numerous 
algorithms for reconstructing a surface from structured light images. For example, the 
phase shift of each line at all points on the image can be determined from a 2D Fourier 
transform. Such a process is described in detail in the article entitled "Fourier-transform 

5 method of fringe-pattern analysis for computer-based topography and interferometry," by 
M. Takeda, H. Ina and S. Kobayshi, JOSA 72, 156-160 (1982), which article is 
incorporated herein by reference in its entirety. The actual surface height is then 
computed by "unwrapping" the phase map. Such a process is described in detail in the 
textbook entitled "Two-Dimensional Phase Unwrapping, Theory, Algorithms, and 

10 Software" by D.C. Ghiglia and M.D. Pritt, (John Whiley and Sons, New York, New 
York, 1998), which textbook is incorporated herein by reference in its entirety. 
Together, a structured light photographic representation of the sample and a 
luminescence representation of the sample may be combined to form a structured light 
superposition or 3D overlay image, with the luminescence data typically shown in 

15 pseudocolor to visually characterize intensity. 

FIG. 5C illustrates a process flow 510 for using imaging system 10 of FIG. 1 to 
obtain surface topography data in accordance with another embodiment of the present 
invention (502 from process flow 500). FIGs. 4B-4H illustrate pictorial representations 
of structured light imaging corresponding to process flow 510. 

20 Process flow 510 begins by imaging a structured light reference to produce a 

pattern without the sample (511 and FIG. 4B). This may be performed by applying 
structured light to a stage or surface that the sample rests upon before the sample is 
imaged. During image capture of the sample, the stage is moved to common locations as 
those used in image capture without the sample. 

25 Subsequently when the sample is in the imaging chamber, the sample is imaged 

with structured light (512 and FIG. 4C). Structured light uses a series of lines of light 
that are projected down on a sample at an angle (at about 30 degrees, for example) to the 
surface normal. The lines bend as they pass over the sample, and the bend in the lines 
can be used to determine the height of the surface at all locations that are illuminated by 

30 a structured light projector (such as structured light projector 170 described above). As 
shown in Fig. 2D, structured light projector 170 is attached to and rotates with light 
transport device 120. In this case, structured light projector 170 consists of a Kohler 
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illumination system where a slide is illuminated by a light source and then an image of 
the slide is projected onto the sample or background. The projection angle is large 
enough to get sufficient "bend" in the lines to achieve spatial resolution, but small 
enough that large shadows are not present. 

5 Process flow 510 then proceeds by imaging the sample without structured light 

(513 and FIG. 4D). This may correspond to photographic image capture (524) as 
described above. The phase shift of each line at all points on the background and sample 
may be determined from a 2D Fourier transform. 

The background data is then converted to a wrapped phase (5 14 and FIG. 4E). 
10 Here, the background data is Fourier transformed and filtered before a wrapped phase is 
calculated. Similarly, the sample data is converted to a wrapped phase (515 and FIG. 
4F) by Fourier transforming and filtering the sample data, and the calculating a wrapped 
phase for the sample data. 

Surface topography for the sample is then calculated (516 and FIG. 4G). In this 
15 case, this is performed by "unwrapping" the phase map. Several unwrapping algorithms 
are available to those of skill in the art for this task. For example, the phase shift of each 
line at all points on the image can be determined from using Fourier profilometry 
techniques. With these methods, a 2D Fast-Fourier transform (FFT) of the fringe data 
(FIG. 4D) is taken to determine the phase shift of the lines everywhere in the image 
20 (FIG. 4F). Since the phase will shift by many multiples of 2n for a typical object, the 
phase exhibits 2n jumps as seen in FIG. 4F. These phase jumps are "unwrapped" in 
order to determine the actual surface. 

The above processes (5 11-516) may then be repeated (517) from different views 
and positions. Imaging a sample from multiple views provides additional information 

25 that helps techniques described herein provide a more accurate 3D surface rendering. 
The multiple images, or the partial surfaces obtained from each view in the 3D imaging 
system, are then registered together to form a complete 3D surface (518 and FIG. 4H). 
Registering can be accomplished by using non-linear least squares fitting techniques to 
minimize the distance between mesh elements on two surfaces that are to be connected. 

30 Typically, the surfaces should have a starting orientation that is fairly close to the final 
registered position. In other words, only fine adjustments to the surface positions may 



be accommodated with this method. Another registration technique is to provide an 
absolute reference line or fiducial of some kind in the image, which gives the absolute 
position of any partial surface with respect to the stage, for example. If the absolute 
positioning of each surface is accurate enough, then the non-linear fitting method 
5 described above can be skipped. 

FIG. 5D illustrates a process flow 540 for converting surface light emission data 
to a reconstruction of the light source distribution within the volume of the object in 
accordance with a specific embodiment of the present invention. Processes in 
accordance with the present invention may include up to several additional steps not 
10 described or illustrated herein in order not to obscure the present invention. 

Process flow 540 assumes possession of a surface representation for the sample 
and possession of light emission data from the sample. Process flow 540 begins by 
dividing the surface representation into surface elements (541). The number of surface 
elements will vary according to the sample surface area and the desired solution 
15 accuracy. The number of surface elements should be large enough to capture photon 
density details and variation across the surface. For example, between about 100 and 
about 10,000 surface elements may be suitable for a mouse. 

In addition, process flow 540 divides the sample interior volume into volume 
elements (542). In one embodiment, each volume element is considered to contain a 

20 point light source at its center. A solid mesh of volume elements then defines a 

collection of point sources used to approximate the actual light source distribution. In 
some cases, the density of the solid mesh increases near the light source to provide 
increased information in this space, while density of the solid mesh decreases in areas 
where no activity of interest is taking place (no light generation or transport). In 

25 addition, as will be described below with respect to loop 556, the volume element size 
may vary during solution attainment according to various adaptive meshing techniques. 
For example, the initial volume element size may range from about 0.1 cm 3 to about 1 
cm , and the final volume element size for volume elements close to the source may 
reduce from about 1*10 3 cm 3 to about 1*10" 2 cm 3 . In a specific example, the initial 

30 volume element size may be about 1 cm 3 , and the final volume element size for volume 
elements close to the source may reduce to about 8*10" 3 cm 3 . 
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Process flow 540 then establishes a relationship between the surface elements 
and volume elements (544). In one embodiment, the reconstruction method uses a linear 
relationship between the source emission strength and the photon density at the surface. 
In a specific embodiment, the linear relationship is described by a Green's function. The 
Green's function contains all of the information about the transport of photons inside the 
sample, including the effects of inhomogeneities in the volume and internal reflection at 
the boundary. The Green's function describes the transport of photons inside the sample 
from each point in the source distribution to the inside of each surface element of the 
sample. 

When the medium inside the sample is assumed or modeled as homogeneous, 
one useful form for the Green's function is a simplified approximation in which the 
surface of the sample is treated locally as a planar interface oriented perpendicular to a 
line connecting a volume element center and a surface element. The photon density at 
the surface is the analytical solution for a point source in a semi-infinite slab using the 
partial-current boundary condition. Since the solution is only a function of the distance 
between the volume element and the surface, the simplified Green's function can be 
calculated for all pairs of volume elements and surface vertices with minimal 
computational expense. 

With a linear relationship between the source strength in each volume element 
and the photon density at each surface element described by a Green's function Gij, the 
photon density at the jth surface element may be approximated by the sum of the 
contributions from all the volume elements: 

p-XGijSi (4) 

i 

where the index i enumerates the volume elements and Si is the value of the strength of 
the point source (photons/sec) inside the ith volume element. 

Light transport in turbid media such as tissue is dominated by scattering and is 
essentially diffusive in nature. The condition for diffusive transport is that the scattering 
coefficient be greater than the absorption coefficient |i a so that the change in the 
photon density is small between scattering events. The photon density produced by a 



source power density, Ui, in a homogeneous medium may be represented by the 
diffusion equation; 

DV 2 p-# ll cp = -U i (s) (5) 
where the diffusion coefficient D is, 



D = (6) 

5 30i A +m) 

In this case, the Green's function is the solution to Eq. (5) subject to the 
boundary condition imposed by the surface of the sample. For a sample modeled as 
homogeneous, a useful approximate solution for the Green's function uses a planar 
approximation at the surface boundary. 

10 FIG. 3B illustrates a schematic diagram showing this planar approximation. A 

plane boundary 312 is drawn through the center of the jth surface element, perpendicular 
to the line connecting the point source at Xi and the surface element at Xj. The photon 
density in the planar approximation is the solution for the point source at Xj in a semi- 
infinite slab defined by the plane boundary, subject to the partial current boundary 

15 condition. In this case, the surface of the sample is treated locally as a simple planar 
boundary oriented perpendicular to the line connecting the source at Xj and the surface 
element at Xj. Specifically, the boundary condition is simplified to the case of a plane 
boundary, although the orientation of the boundary may change for each surface element. 

This simplified Green's function is the analytical solution for a point source in 
20 the semi-infinite slab using the partial-current boundary condition: 

* 

1 fexp(-n eff iij) i i ] 

znu { ijj z b z b j 

Here ry = |xj - xj| , Ei is the first order exponential integral and 
Heff =[3Ha(Ha+Hs)1 ( 8 ) 

C 1-Reff 
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In the simplified model just described, the simplified Green's function depends 
only on the distance between the volume element and the surface. It is not necessary to 
use an analytical form such as the simplified approximation to define the Green's 
function. 

5 The present invention does not rely on an analytical form such as the 

approximation described above. In another embodiment, a look-up table may define the 
Green's function. The look-up table may be created by previous measurements of 
. photon transport in a sample (or similar sample approximated to be substantially equal to 
the current sample), or by computational simulations using techniques such as Monte 

10 Carlo or finite element modeling. This particular method is useful for samples 

consisting of inhomogeneous media, such as animal subjects. In this case, the optical 
properties, andm from Eq. 8, now have spatial dependence. 

The planar boundary approximations discussed above work best for smooth 
surfaces with a large radius of curvature, and for cases where the absorption coefficient 

15 is not too small (/4 > 0.1 cm' 1 ). An advantage of the planar approximation technique 
described above is that it is computationally convenient for solving the diffusion 
equation with an arbitrary complex boundary such as a mouse. Areas with more 
structure, such as the head or the limbs of a mouse, may benefit from a more accurate 
model of the boundary. Using a finite element modeling code to calculate the Green's 

20 functions is one option to obtain a more accurate boundary model. Finite element codes 
such as Flex PDE, from PDE Solutions, Inc. may be used for example. Another option 
will be to extend the planar surface approximation to first order in curvature, which may 
allow continued use of analytic expressions for Gy. 

Once the Green's function is determined, the reconstruction is obtained by 
25 solving the system of linear equations that relate the photon density at the surface to the 
source distribution inside the object. Process flow 540 then proceeds by solving for all 
the internal volume elements (546). More specifically, given the modeling described 
above, the reconstruction techniques solve the system of linear equations that relate the 
photon density at the surface to the source distribution inside the sample. Thus, once the 
30 Green's function is determined, it may be evaluated for every volume element - surface 
element pair, in order to obtain the system of linear equations (Eq. 4). The final step of 



the reconstruction method is to solve the linear system, Eq. (4), for the source strengths 
Si. Referring back to Eq. (4), since p is known, and Gy can be determined as described 
above, the reconstruction techniques then solve for Si. Typically, there is no exact 
solution to the linear system because the collection of point sources is only an 
approximation of the actual source distribution. One suitable reconstruction is then the 
best approximate solution of the linear system. 

In one embodiment, the linear system is converted into an optimization problem 
to find an approximate solution. In this case, a cost function is used to define the 
optimization problem. One suitable cost function may be represented by the sum of the 
source strengths: 

Cost = £Sj (10) 

■ 

J 

The cost function is subject to one or more linear constraints. A first suitable set 
of constraints is that the source strengths be positive definite: 

S t >0 (11) 

A second suitable set of constraints is a system of linear inequalities based on the 
system of linear equations (4). It requires that the contributions to the photon density 
summed over all the volume elements be less than the measured value of the photon 
density at each surface element: 

XGijSj<Pi (12) 

■ 

J 

In a specific embodiment, an optimum solution for source strengths Si is found 
by maximizing the cost function (10) subject to constraints (1 1) and (12). 

Process flow 540 then solves for the current optimal solution in the set of 
solutions (550). The optimization problem described above may be solved using linear 
analysis techniques. One particularly efficient technique is the SIMPLEX method. 
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The solution quality may be assessed (552). In one embodiment, the assessment 
measures the difference between the observed photon density and the calculated photon 
density. For example, a "chi squared" criteria may be used: 



* 

J 



Pj-SGijSi 



(13) 



5 The value of x 2 measures the difference between the observed photon density Pi and the 

EGijSi 

calculated photon density » over the surface of the sample. 

Determination of internal 3D light source distribution data based on surface light 
emission data may be embedded in one or more iteration loops. Iteration may allow for 
refinement of the volume mesh and may allow searches for the optimal sampling of the 
10 surface elements and the optimal weighting function configuration. In general, the 

convergence of the reconstruction is improved by reducing the size of the linear system. 
In one embodiment, reconstruction techniques of the present invention iteratively vary 
one or more of: the volume element configuration, the surface element configuration, 
and/or the cost function configuration. 

15 Iteration may also assist solution attainment. The SIMPLEX method converges 

rapidly to a solution, and hence has the advantage of being a computationally efficient 
algorithm. To assist the SIMPLEX algorithm in finding the global maximum, and hence 
the optimum solution, the SIMPLEX calculation may be repeated numerous times while 
systematically varying key elements in the algorithm (554). This can be achieved by 

20 using adaptive meshing, or adaptive gridding, to vary the number of volume elements, 
and by using a subset of the surface elements. 

Loop 556 (FIG. 5D) illustrates repeating the solution steps while varying the 
number of volume elements. Typically, it is advantageous to reduce the number of 
volume elements in the problem while maintaining a high density of volume elements in 
25 the vicinity of the source. This can be achieved by using adaptive meshing. In one 

L 

suitable adaptive meshing application, a coarse volume element mesh is initially applied 
throughout the entire sample volume and the current solution is found (544, 546, and 



550), yielding an initial solution for Si. Next the volume elements that have source 
strengths greater than zero (Si > 0) are refined (i.e. subdivided) and those where the 
source strengths equal zero (Si = 0) are removed. Solution attainment and volume 
element mesh refinement may then be iterated repeatedly, producing a high-density 
5 volume element mesh localized in the source region. During each iteration, the quality 
of the current solution is assessed (552). In a specific embodiment, the iteration 
continues until further refinement produces no significant decrease in the value of % 2. 

An additional iterative improvement may be obtained by varying the number of 
surface elements, N s , used in obtaining the three-dimensional representation (loop 555). 

10 Using a subset of the surface elements of the surface mesh reduces the number of 
constraints in the problem, which may simplify solution calculation. The number of 
surface elements may be used to sample the surface uniformly. In this case, process flow 
540 iterates for different values of N s corresponding to sampling the surface element 
mesh at different densities, and use the quality assessment (552) to determine the best 

15 solution among the different values of N$. For example, if the number of surface 

elements is between about 100 and about 300 surface elements for a small mouse, an 
iteration step size between about 10 and 50 may also suitable. 

Another iteration loop varies a cost function to improve solution obtainment. For 
example, the cost function may be modified to include a weighting factor W*: 

20 

Cost = Xs i /W i Y, W^Gy (14) 

* » 

1 J 

. The weighting factor Wj is the contribution of the i th volume element to the photon 
density over the entire surface. The exponent y adjusts the relative contribution to the 
cost function of the interior volume elements and those volume elements close to the 

25 surface. When y = 0, then the interior volume elements have relatively greater weight. 
When y = 1 the volume elements near the surface have greater weight. Process flow 540 
may be iterated while varying y to search for solutions where the source is both near and 
far from the surface. For example, the step size may be varied by about 0.01 to about 0.2 
for a range of y from 0 to 1. In a specific embodiment, the step size was varied by about 

30 0.05 for a range of y from 0 to 1. Once again, quality assessment (552) may be used to 
identify the best solution. 
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In a specific embodiment, the SIMPLEX calculation is imbedded in a search 
routine that consists of two optimization loops. The inner loop uses adaptive gridding to 
optimize the configuration of the volume mesh. The outer loop searches for the 
optimum values of the parameters Ns and y. Each step of the search algorithm is tested 
5 using the chi-squared criteria. The global solution is defined as the one that minimizes 

x 2 . 

Figures 5E and 5F represent reconstruction results for one point-like light source 
within a phantom mouse (e.g., rubber mouse having an embedded fiber optic). Figure 
5E illustrates the reconstructed source distribution within phantom mouse 702a and 702b 

10 at two different angles. The surface shown represents the contour of the isosurface at 
half the maximum source strength. Figure 5F shows images of the photon density (e.g., 
708a and 708b) directly beneath the surface of the phantom mouse 702. The first row 
706a is the logarithm of the measured photon density. The second row 706b is the 
logarithm of the simulated photon density, calculated from the reconstructed source 

15 distribution. The third row 706c is the difference of the first and second rows. 

Although process flows 500, 510, 520 and 540 have been described with many 
simplifications to expedite processing, the present invention is not limited by these 
simplified computational methods. For example, the Green's Function may be 
calculated without many of the simplifications described above, even at the cost of 

20 increased computational requirements. In addition, while process flows 500, 510, 520 
and 540 describe a specific method of obtaining measurements of light emission from 
the sample based on the system illustrated in FIGs. 1-2E, 3D reconstruction techniques 
of the present invention are not limited to how the 2D light emission data is obtained or 
to the use of any particular apparatus. Indeed, other imaging apparatus other than that 

25 described above with respect to FIGs. 1-2E may be used, such as an imaging apparatus 
described in commonly owned pending patent application Serial No. 09/905,668, which 
was previously incorporated by reference. In addition, 2D light emission data may be 
obtained from an independent source and stored as data within computer 28, and not 
necessarily produced as the result of imaging via a complementary or local imaging 

30 system. 

Further, although the present invention has been described so far with respect to 
a bioluminescent source, the present invention may be used to obtain 3D reconstructions 
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of any type of light sources, including one or more fluorescent sources. For a fluorescent 
source, the relationship between the surface elements and the volume elements may be 
altered (544). For example, a Green's function in the linear system may be multiplied by 
a second Green's function that describes the transport of the excitation light from the 
5 sample surface to the volume elements. In a specific approximation, the Green's 
function in the linear system (4) may be modified to be the product of two Green's 
functions: 

Gy=G?Gj (15) 

10 

The first Green's function, Gj E , describes the transport of excitation light from the 
excitation source at the surface of the sample to the volume element. The second 
Green's function, Gy , describes the transport of the fluorescent light from the i volume 
element to the j m surface element. Both Green's functions can be determined from 

15 analytical expressions, such as the simplified approximation described above in the case 
of a homogeneous medium, or from look-up tables in the case of an inhomogeneous 
medium. The excitation and fluorescent light are typically at different wavelengths, and 
thus the fluorescence does not stimulate additional fluorescence. The system of linear 
equations (4) is still valid, and process flow 540 can be used as outlined above to 

20 determine the fluorescent light source distribution. 

The present invention is also well suited to handle noise in the input images. 
Any practical imaging system will have a limited dynamic range and be subject to noise 
sources. This may compromise the fidelity of the images used as input data in the 
reconstruction, and consequently may degrade the quality of the 3D representation. To 
25 this end, the present invention may also include techniques designed to improve its 
performance when the input images are noisy and/or have a limited dynamic range. 

A limited dynamic range is particularly evident when imaging bioluminescence 
from sources imbedded in tissue, because the light emission intensity typically varies 
over many orders of magnitude across the sample surface. If the imaging camera 
30 imposes a limited dynamic range and a region of highest intensity is set to the camera's 
upper limit, then there will probably be regions in the image where the emission 
intensity falls below the bottom of the dynamic range. These regions of the image will be 
received as noise; and correspond to a 'noise floor'. 



The reconstruction techniques provided herein map images of surface light 
emission onto the three-dimensional sample to determine the value of the photon density 
underneath the surface. With noise in the images, the reconstruction techniques map 
only those regions of the images that are above the noise floor. This can be done by 

5 defining a threshold for each image that is a fraction of the peak intensity. In one 
embodiment, the threshold is related to the peak intensity and dynamic range of the 
camera. In a specific embodiment, the threshold may be larger than the number 
represented by dividing the peak intensity by the dynamic range. For example, if the 
dynamic range of the camera is 1000 and the peak intensity in the image is Ip, than a 

10 suitable value for the threshold may be I P /500. Only the regions of the image that are 
above the threshold are mapped onto the sample; and those regions below the threshold 
are not used. As a result, there will be regions of the sample surface that are empty after 
mapping is completed, that is, that have no photon density values associated with the 
surface mesh. Typically, these will tend to be the parts of the surface that are the farthest 

15 away from the bioluminescent source. 

Process flow 540 solves the following optimization problem: maximize the total 
source strength subject to the constraint that the calculated photon density is less than the 
measured photon density everywhere on the surface. However, process flow 540 may 
also account for empty parts of the surface. This is because when the photon density is 

20 unknown, the constraints on the source strength are removed for those volume elements 
near the empty regions. Thus, there may be insufficient information to determine the 
source strength in those volume elements. The modification of process flow 540 
consists of eliminating volume elements in the volume grid that are insufficiently 
constrained by the existing photon density information. This is achieved by comparing 

25 the contribution of a volume element to the photon density in the empty and non-empty 
surface regions. For example, if P and Q represent the sets of surface elements that 
consist of the empty and non-empty regions, respectively, then the criteria for removing 

th 

the i volume element from the volume grid is: 
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The constant k may have a value in the range of 1 - 10. The criteria (16) is applied to 
- each volume element during the formation of the initial volume grid (542) and at each 
iteration, if used. 

In addition to having a limited dynamic range, realistic images may contain 
5 noise. One minimum noise contribution may be represented by the shot (Poisson) noise 
associated with the sampling of discrete numbers of photons in each image pixel. When 
shot noise dominates, the signal-to-noise ratio in the image varies with the square root of 
intensity. This suggests that the regions of highest intensity, or equivalently those 
regions of the surface having the highest photon density, should have the relatively 
10 smallest amount of noise. To account for this noise, the assessment for solution quality 
may altered (552). For example, expression (13) may be modified to account for noise. 
More specifically, each term in the sum may be weighted by a factor of the photon 
density pi to become: 

2 

(17) 

15 The reconstruction techniques of the present invention will typically be 

implemented by a suitable processor or computer-based apparatus. Referring to FIG. 6, 
an exemplary computer system 350 includes a central processing unit (CPU) 352, read 
only memory (ROM) 354, random access memory (RAM) 356, expansion RAM 358, 
input/output (I/O) circuitry 360, display assembly 362, input device 364, and expansion 
20 bus 366. Computer system 350 may also optionally include a mass storage unit 368 such 
as a disk drive unit or nonvolatile memory such as flash memory and a real-time clock 
360. 

Regardless of computer system 350 configuration, it may employ one or more 
memories or memory modules configured to store program instructions for obtaining a 
25 three-dimensional representation of a light source located inside a sample and other 

functions of the present invention described herein. Such memory or memories may also 
be configured to store data structures, imaging data, or other specific non-program 
information described herein. 
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Because such information and program instructions may be employed to 
implement the systems/methods described herein, the present invention relates to 
machine-readable media that include program instructions, state information, etc. for 
performing various operations described herein. Examples of machine-readable media 
include, but are not limited to, magnetic media such as hard disks, floppy disks, and 
magnetic tape; optical media such as CD-ROM disks; magneto-optical media such as 
floptical disks; and hardware devices that are specially configured to store and perform 
program instructions, such as read-only memory devices (ROM) and random access 
memory (RAM). The invention may also be embodied in a carrier wave traveling over 
an appropriate medium such as airwaves, optical lines, electric lines, etc. Examples of 
program instructions include both machine code, such as produced by a compiler, and 
files containing higher level code that may be executed by the computer using an 
interpreter. 

It should be borne in mind that although computer system 350 is discussed in 
some detail herein to facilitate discussion, the invention may be practiced using a variety 
of suitable computer-implemented techniques. In general, any suitable computer system 
may be employed for obtaining a three-dimensional representation of a light source 
located inside a sample. Further, the inventive reconstruction techniques disclosed 
herein may be implemented via a computer network, such as a local area network 
(LAN), wide area network (WAN) or a global computer network such as the Internet. In 
the latter cases, the reconstruction techniques may be implemented at least in part as 
downloadable computer software and data (e.g., applets such as JAVA™ applets from 

r 

Sun Microsystems Inc.). The downloadable computer software and data may be kept on 
one or more servers on the network, accessible by any client computer or terminal 
capable and authorized for such access. Network computing techniques and 
implementations are well known in the art and are not discussed in great detail here for 
brevity's sake. 

While this invention has been described in terms of several preferred 
embodiments, there are alterations, permutations, and equivalents which fall within the 
scope of this invention which have been omitted for brevity's sake. For example, images 
may be acquired and analyzed at one or more different wavelengths. It is therefore 



^rr^*%.Ts-\T^f\rin m~\j~f- nil tto 



1") 



intended that the scope of the invention should be determined with reference to the 
appended claims. 
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